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trum £t tliQ NgtKer firaquency end tfie power &|>6ctiuifi 

belwMn peakB b detemilned. Jh^ eantr&utlafi to tiw 



ampiltiido olp«aks at the oommon frequency Interval by 

svbliaotod tontllio aniplituda of tfia poaUcs at com- 
mon rreqiioncy miciiva). vm nvm^&i or iho corrocf^d 
amplibjdes of tba peato atttiecoounon frequency intar- 
vai l» obtaSnod «fti(f noism^lliod m a tkeklne^s motde 
forthovkfeoaigiiaj. 
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BACKGROJND OF THE INVENTIOM 

[OQtM] The present cnventlon relates to ^he asseBS- 
ment «^plcturo quality ^or video Imago sipals, and more 
pa/tlcuiaily to a hannonic method and appaiatua for 
measurtng Wocklfvcss h video Imago signals. 
[0002] In dfgial vid6o transmission whem brnic^width 
i8l»portfim, such as uansnnlsal^n over ^te11lt« link, th« 

sion to decrease the bandvsidth required for each video 

duinndL The comipfes&lon siandaids ^icaiiy tiEetli 
wrcti » JPEG, MFBQ or prpprieiarjf varfaBta ihereof, 
aro *tessy* *• to adslovo higher compression ihcy allow 
distortionB to oocurm en image reprBsentedby the video 
signal Ihs emount (^^tortion Is a function of the oom- 
pldxlty o^ihd im^o and of tho numbor of bits sdcond 
(bit rate) a comprcssl^ encoder Is allov^ to uso. Ido- 
aiiv Old mm*i or eomnrdssion id rti&ximi2dd m\Q m 

providing a video Ina^o or pkmre lo the cif^tomer that 
is Um ol dlsiifiblng artifacts. 

[0003] Cuirent devices to analyze picture qual^, 
sttch as iho A200 Piciuro 6uafey Analyzer ffiamjfac* 
lured by Tektronlac, Inc. c^Beaverton, Oregon, (iSA, are 
rofcrcncc>basod. A victoo ^nal titMtsiTtltcd Ihroufgh a 
Video sysism t& con^ifirsd wnii ths ofigmsi vic£36 ssgnal 
aa e reference video s^nat fi) a measuren^ent device. 
Tb6 rdferdnce viddo signal is e^ior stc^dd In ths meas* 
urcmcnt dcvipc m ^ irsn^gfitcd via owno other nofi 
distorted patfiway to (he measuremefil dev&rd. Use of a 
reference vWeo alginel b necessary for ex!remely accu- 
ralo a^orithmSi such as tho Samott Corporation JND» 
metrix'** human visual model algorithm. Hoivevei, this 
means that measurements are only made on vJdeo sig- 
nals whoso oontenis dn» o^6<r l^own In dxmnco or ard 
immediately avaiat>b. sucfi as csouibic^lieckiiiig an en* 
coder ai the souica. 

(0004] Otiter potential method^ ol measurlrtg discrete 
cosind transroim (OCT) based OKfac dsgiadadons In- 
volve directly examining the cotarsenees of tlie quaati- 
zaiton scale In tho tOAtprcssod vidoo stream* optionally 
combined w\&t a measure of the complexity of the ori^ 
Inal Imago tmi^mittod by some means outside tho video 
ctonnsi - a form of compresBed rsfflrenoe. Tliia method 
IsRotasaocurate. ajKttnanycaseoanontymakemeas- 
uremonis on oompressod vtdso. not virfeo thai has been 
already decompressed and pptentlalty passed titraugh 
other systems, Induding ooiar addiisonal codec$, prior 
to end-uaentellv&y. 

[00O5] V\matl8de^edl&amGthodanda^araiusthat 
allows measurement of blockiness artlfada, euch as In- 
cttf red v4ton compressing vtdoo ttslng discrete eosino 
transfosrmSy in situations where Ihere Is no ptfor know(* 
odgo ol tho btocMnoss period and in mo picsenco or 
Image^ontent generated noise. 



BRIEF sgMMARY OF THu m^mm 

[0(MI61 Accordingly the present Ifwenliwi proindes a 
hamionte method and apparatus for measuring biodd' 

9 ness In a video input signal A power spedium is 'Ot>- 
talncd for a v«eo ^IcW of the video Input ^nal as the 
Fast Fourier tr ansfonn Qi the sum of the lines m the video 
line after edge fiiterfog. The largest frequency peaks 
from the l^her frequency end of the power spectrum 

10 are initially a&lected, and a eommon perlodfortHe peaks 

is del^M repre&6ntihg & commoh frd^ueii^- inldif* 
val The ampSUfdes of the peaks al the common fre- 

quancy imarvai are cofrectad for tho oantributlons made 
by peaces at one-^wlf and dtnible the common fretpency 
>s intervaL The corteacd an^lliudcs of tho peaks at tho 
conmor) frequency JrvtervaJ are averaged and normal- 
ized to prodwe a blockiness melrfo for video In put 
s^nai. 

[0007] The objects, advantages and other novel Pea* 
^ ]um or iho prdsont invsntion dro apoaront from tholoi^ 
lowing detailed description when read In oonjuncsion 
wtth tho appondod cla^ and attadisd drawing. 

S5 THEDRAVyihta 

[Q008| Figs. lA-Daregrapniieviauiaoftnospocinim 
of a video signal wlthoot snd widi blSoddness and wliti 
spurious peaSKB. 
» jqooaj _Fip.2!5aflDW<^ftvlcwofa5]^ 

uting blockutess nn video s^nals according to the 
present Invention, 

[0010] Plg.3isablocktigs^ramvtowofaflap)»araitus 
for measuring blDddnass in v&leo signals aooonfing to 
99 tttoprcsomlnvomlon. 

[0011] The basis lor tlte preaeni Enventioft Is tlie •ob' 
^ ssnridion that spsctm of vddao fields tnpalred mritli 

faloddneee artifacts have a charBctaiisliiB fonm, as IIBua- 
tratod In Figs. 1 A*D. Such spectra of a vMoo fIcSd ^aro 
obtained by edgefitteftng each line csnying video Itmi^ 
oontont, sunnming allthovldoo fines rram one Wdoofleu 

<^ into one line, and then taking a Fourier transfonn, sucli 
as a Fast FoinlerTramiform (FR), of the resting l&te. 
[0012] F^ 1 A represents a lypical spedrvm of a vid- 
eo field that Is produced by vWeo coraem In the absentee 
of blockiness artildcts* TItSs spectrnm varies s^htly 

^ tlromfle(diofleld,dependlnguponGonieni,butiltebasic 
naiuro of Its shapo Is preserved. Fig. i B shows tho typ- 
ical apectnim where bloddneEa aitifacte are present si 
Oto vIdoo. Hero tho peaks are located on Imcgor multi- 
ples of frequency wftich is the red^ocal of the period 

« ym whfeh tho block boundaries occur. Ideally thopoaks 
are all ec|UBl in magnludb, but the effecte of ImpQcIl: fil- 
terlng may reai^ In shspea such es ISualrated in Rg. 
1 C. Video content often is rich In Images with sharp ^dg- 
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OS or wtifeh car^ flrtlfa^e (ncwrrosi from fjnl?o jmpvteo 
responsa (F(R) cf^itaJ lillering. in theso Eitadik>n& thd 
^uailon jlltmlreted In Rg. 1 D mey result, wtiere various 
pcaKs In mo spcctnimarc present along with peaks mat 
represent the blockinsss artifacts. 3 
[0013] What fs shown ^ R9. 2 Is a rm^od of allowing 
renrovBl of spurious frequency peo.'ka in the ^actnim 
frflw tlio mca$urcm<mt an<f mm allovrs ^ntllteatlon 
dnd measuromonl of spectn^ conldsit gonomted ty 
btocklness ^i^m. The large pealcs ffm tha higher fr^ 

^ueitey end of the spdetfwi) dre eorldd frdni Snidng Ute 
spectral peaks generated, and those spectraJ peatcs that 

dd mi rail onto stteger muttpies or a fi^^uonciy Iftiaival 
are rejected, Dy doing tha, most jptirlpu^ peaS» m Ihe 
specJrum arc rcjcctod ami onfy mo pca*$ goncraicd ty w 
the bbckmessBrtifad remam. The amount of hlockjnees 
preeem &i th« vtcteo then determined as some aggre- 
gate m^QAm^ derived rrom those spectral pea3(d, one 
moasuro being tho average of the ampiluctes of 
the o^m, 30 
{tm 41 Once the spectrum of the aggregate Sne Is ob- 
t^ncd, It is analyzed to dotomiino tho prodomlnant 
spedrat peeiks and whether these peaks are regubrly 
s|?acod» tndbatkg bJod«^c^ The to N discrote Po«- 
rier transform pFT) coetficisnts, where N may equal SO S3 
for a 512iJ0fnl DFT, that e^rcspond to tho lowest fro- 
quflfu^ oms wiier^ most qi me nmm pmif& ensi^ la 
located are discarded Then the spectrum la flered to 
obtain a «mooShdd baseline specbum which Is £td>tract'^ 
c() frcHi) ttio orlgfuQl la obtnln n noimdizcd opootriim, ^ 

Tlus removes most noise, such as white n<^, and pio- 
twre content energy from U« ^ecmim. Then the targest 
P pcaJos In mo spectrum are found, and this sot Is 
searched fore common p^iod.ThecDrnroon period cor- 
responds to the predominant blocldncss pcflod in iho ^ 
Im^e. 

{661^ Once tho predominant bloeldncss porlotf k do« 
lemoned, the aneuni of btoc^ne&s fn thatpariod is cai- 
odated. To do ihia the av^iage hannonia pe«t3t heighit 
after raniovlngtha estimated contifteitions to the psalcs ^ 

from half and double btoclclnesa period, is measured. 
Tftcn the b^ooklncss amotintis soalcdby iho tteeklncss 
per^d to obtain e normalized bloddness metrfcthat ma/ 
bo oomparod across dltforom poHods. 
[001 6] QsneiBily bOoddness has horizontal and verti- 45 
cat components that arelndepen(fem.A8afradeoffo<nly 
the horteointal component is compded above. To calcu* 
late tho venical component, the horizontal blocMness 
detection above {& run on the transpose of the original 
Inpui Image. Ideally a bSockilne^ (mpliemematlon com- » 
ptjtds tho spectfum ol aach lino and aggregates by av- 
era^ng^ rather than aggregolEng fast and computing the 
spocvunx This provonis tho phase ohametortsttcs of dlf- 
ferant linos from dSstorting the aggregate spactnim, i.e., 
canooling each other out bcforo iho magnltudo ts cafcu* " 
lated. But tfib technique currently is prohibttivefy expen- 
sive, 30 a leas lebust rmpSementatton of measurthg the 
blockinsss En fitsi one (horizontal) direction Is used. 



1 > Tho ampHtudoe of the spootr^^ ooettislonts l&e> 
h««ent'^vo frames vAth 6<)ual dmounls of biockinets 

m pioponitml to the bloc^ln^s period and the 
frame with the longer period has propoftionaliy 
more peaks in its spectrum. So if frame 1 has a 
btockfiK^ period of 6 and f rsmc 2 has a bloodiness 
period of 16^ frame 1 has half as many spectra] 
peaks at twfeo the arripHturfo than the spcctraJ 
peaks from frame 2. The metric used to quantify 
bSoddneis ra^es th& fact into acooum by nomiallz- 

Ing the tiEoadness rdsull 1^ IhO blOtluhdss pendd 

2. if a frame contains mu&lple blockinesa perkxis 

thai Share commor) harmonics: \.^xi 8 and 16 J and 
12, etc,, the aggregate speoium Is distorted. IFor 
tho 8 and 16 case, for cxamplo. aQ of the spectral 
peeks fortheblockinese period of d are increased 
by the c^ntr^utton from the blocklness period 1$ 
energy. Without accounting for lhas possbiliiy. 
greater btockiness period 8 energy m«y bo compirt' 
ed than is mum Pfe£«nt Convorso^, to ftisasurtt 
the bfocklnes$ perfod 1 6 spectraf peaks, even sp ec- 
tral poaks which coincide with period 8 peaks .are 
greater than odd peaks. This suggests the fcilowing 
way to estimate the rcMe oontribuilons of period 
S and period 1 6 Uoddnesa energies: 

GWcn tho dcs&od bfookinoss period P to CS' 
timata tho fpecsral eneigy, ma avsrage eneiigy of 
the odd hainH»nk»i of perfod S^P la eomptited «nd 
stAtracted from aSi the peaks of peilod P. The cur« 
im liip^ntstlon pcntc^g tliis firm aider oorroo 

lion for the potentially common <P« 2P} second har* 
monjo case, but not for the moie general (N^P, M *P} 
case which trkts to aocount rorerbitraiy shaied fiar* 
monic conf igiiiBiions. 

3. (fthc goatistocstoutatothobiocldnossincftrlcat 
a known period^ thore pdartisaily is aniinprovemient 
h ofttelcflcy by fust oaleufatf ng DP? coeifidonts 
ai the de&^ad frequencies dtroctly mthar than cal* 
cutallhg the entire spectrum. For exampie for e 
UDdt£ties&of8«onlythecoercieiitsat64j28and 

ISS need be calculated fora &t2-paint OFT In^le- 
mentailoo. Tho spectral eoofMcnts groater than 
25$ nmor ttwjse less than 256, 

4. Sooauso 010 Oft cateulatos onty iiuogral Ire* 
quency bins, some emor fis introduced when fiar- 
mofilcs fall on fractbnat frequency boundatles. Ei- 
ther a greater resolution may be calcutaled, le.. us- 
ing a 1024^oint OFT with zero padding, or scene 
form ofinterpoisaiton. But with inlerpolaiion spednsS 
smeartig ooguts, causing several of die nelgtibor- 
1^ bins tn the DfT to roeelve energy from the frac- 
tional hamioidc. The smaller the number of points 
tn tho DFT, the moro the accuracy and precision of 
tho blocklness meln'c degrades. 

[0017] Whtle the method described alravB derives; an 

aggregfiia measure of bfoddness of the whole frna^e, 
the method may be applied to local radons of a Wdeo 
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intago and the aggre^ato moasvro o{ b^oeldnosa may 
bG obtained by composing it from local measwes of 
blocklness weiglited eccofcSng lo a human vl^on fiiter- 
ing oS Oio undcMyinQ Imago. 

[001 8] Belamng now to Rg. 3 the appBrntus rsquired 
to Implcmeni tho metlwd Mudcs b video s^nal rcootv* 
er 12 which converts the video signal to digctat form so 
II may bo stored In an array orb)t-n\«p In computor mcm^* 
dry 1 4, and a ntor 1 6 to obtdin the spectra m descsribed 
abovo. Spect/al p«ak Mt^mmUm p«rtaifttng tfi Nock^ 

nass Inxpdimienl Ihen is i&dfated^ ^ deEcin^d aboV^* 
by an appfoprfate anslyzer 1 8 and the blocklness m&aa^ 

am 18 feponad on an ai^rop^ate (gsplay 20. 
{0019] Ihus the prwent iivention pmt^^i a mew 
uro of t^of^css using frequency harmonics by gener- 
ating fronn an input ^bo signal a pov/erBpec^rrir se-* 
lecting the largest peaks in ttw spectrum, rejecting those 
im. are not intdger muUip^ds ef a ft^uency. linding uio 
langosi b{od(£nes$ amount oof rcsponcilng to cad) set of 
09m, m mpoMn thd bioekinosft uined and mmtA 
for the dlQtnn^nt set of pe«k)e. 



1. Amc^odofmoastfrlngbloeXfitcssinanlnptftyideo 
signal usJng tiaimof!^ e^pmmg m& steps <^ 

gaiterating a power ^pedrum for a >^ddd fierd 

Qf^he input video olgmsli 

««iecttng a set of peaks from the pwm tpo> 

tnim ttiat fsl! at a comnion fneqaency Nerval; 

and 

detennlnlng a blockineas maesura for Ihe eat 
ef poaiGS. 

The mmo4 as recited In dairti i wticrein tlie gen* 
ef aUng &tsp comprfsea iha steps of: 

fitting m vScfeo mi lo accentuace ed^ 

summing the Imos of thefStered video fialdbiio 
a Single line; and 

pexfeimlrtf a f ouner tmnsferm o^a Hie 
lino to proctoeo the power ^cctrum. 

3. The memod aa recited &t elalm 1 whereto the se- 
todlnd etap cocxtpifsos the ^epd of: 



dfecarding the m\ N ^ilacrolo Fourtor tranelGrm 
coefficients ref^asenUng tha content of the vid- 
€0 field eorrespomjing to the lowest freqiiency 
bins: 

5 filte-ring tha po^ver spednim to obtain a 

smcomcd baseline spectrum; 
&ti>tractlng tha smootined baseline spectrum 
Ifomihc power ^Kjctrunitoobtalnaaormallzod 
spectoiK and 

io lir^dlng the larg&st P peaks In the normalized 

ip^iim &s (IFie milial set of |^kg. 

5. The meihod as rscitad in claim t \%tiare^ me deiar- 
mining alep conipri^s Ihe step of averaging the 

)^ amplitudes or the sot or peaks as tlio blockiacss 
measure. 

6. The me^od as recited «n da^ 3 Wherein the delete 
mtnlag stc|> eomprisos the steps of: 

removing cofitrijutlons to tfie ampiudes of the 
set of peaks adtted t^ peaks at <»to-iialf and 
datibta the common perled; 
nieasur^g an average itarmonb podc an^ll- 
23 lude of the peaScs ai the common paccod lo «eb> 

tain a biocKinoss amount; and 
scaBng ms mmtmSi fimount by tue eommoii 
pef^ to <^tain a normalized bloddnesa mefiHc 
as the biQciclndss nieasure. 

7. Th« memod as foeied in daim 6 iivherein the pb^ 
mov1ii9 at^ comprises the steps ef ; 

averaging tha anplltudea of peaks at the odd 
39 haimofiics of twice ^o common period to pr^ 

ducean aver^annplitiideeentfibution; and 
stibtf acting the avx^ragc amplitude eontrlbusjon 
from the amplitEfdes of tha peaks al the com^ 
nion peiled. 

8. An appacatos for measuring Moddness in an in put 

vkteo Signal using hamionlcsoompHsingmcafiSifor 
peneraitinfg a power spectrum for a video field of the 
li^ut video signal, moans for sdoctlng a sc& of 
^ peaks fn3m tlie powar spectrum that faO at a com- 
mon fiequency intefvaf, and means for determining 
a blodclness measure ef the peaiks. 



sorting large peaks at a higher frequency end 
cftheponrerspetstrumtoobtalnanlnftfatseiof ^ 
peaks; and 

searching the initial set of pe^ to find the set 
Of peaks havfirtg a common pctled bcivveon iho 
peaks, the eommon perfod defining the com- 
mon froquoflcy IntotvaL ss 

4. Themfi(hodflareeitedlne£a]m3where1nthesonlnd 
step comprises the steps ef: 



4 



BEST AVAIUBLE COPY 



EP1 111 928 A1 



8)^ 



at 

J- CO 



I I I I I m rl l" l I I I TI T I I'T I I I I 

SPATIAL FREQUENCY 

./PIXELS, p,^,_,^ 




1 1 I I I If 'f n > 't' M I Y M I V I 1 1 'r I 



SPATIAL FREQUENCY 

<1 /PIXELS) cir* 4B 

FIG.1B 




I M 1 1 iT'f n I V 1 1 rr 1 1 I v I I I'l' 1 
SPATIAL FREQUENCY 

(1 /PIXELS) cir^ tr- 

FIG.lC 




SPATIAL fftEQuENCY 
(1 /PIXELS) rxr^ 

FG.1D 



5 



BEST AVAILABLE COPY 




EP1 111 92SA1 




:0 



I 



St: 



UJ 



I 



CO 

o<o< 
m 



^ ^ . o 




N»0 



3 

a 

' z 

o" 

U 
O 

> 



<a 

Oq 

op 



UJ 

<UJ< 

i-Oui 



<z<o 



Ul 




o 



6 



BEST AVAILABLE COPY 




EP1 111 as^Ai 



EuMpeanlHrteni 
men 



EUROPEAN SEARCH REPORT 



El> m 31 1329 



DOCUMENTS CONSIDEflgD TO 8£ HEIEVANT 



Km H ET All "A nm nmm of «EASURifiS 
THE BiccxiF^ erfccTs OF imsES BASED m 
cmmi immjim' 

lElCE TRA»SAm(HS5 ON rUlffiAflEI«TALS OF 

EtcafiflNics, ccwifUNiCATmJS Am mmjik 
minas^^p^mnnn of ELECTwmics 

IHFOl^TlCfl m (m. TOKYO, 
vol, 8» 
i AiiQitst iim-m-U), pages 
1274-1282, XP0Q8633543 

^ m wHdU aficuRiefii * 

HUAk^ J £T At: 'SLOCfCINO AmFACT mmkl 

mm ON FSiipNa AmisiS' 

£L£CTftONICS irrtlRS.IEE SIEVEJWOS^OB, 
voK 34, no. 24, 

2S IJov^feer 1998 (1958-11-26) , phages 

2323^2325. XPO^BShaiT 

TS^J: 

^ tM whole docufneni * 

GB 2 347 ail A (rUDCPiilQSJT TCLIViSrON 
CmnS} 13 SeptefDber 2000 (2^09-13) 
* the dficument * 



Ihe ^rasent OKirch N)9 baaniinawi for si) cistns 



CLASSJ nCATlOM OF THE 



1*S 









SERliN 


9Harcli 2001 


Ones. T 



CAU^CCRY Of OTEO OOOMENTS 

Cttumorc el m»«*TW csttgory 

P: 



dUftflttAtftXldllB 

O :^euneni>ctoJki Bioat i:»:atlqit 



7 



BEST AVAILABLE COPY 



EP1 m^Ai 



ANNEX TO TIIE EUROPEAN SEARCH REPORT 
ON EUROPEAN PATENT APPLICATION NO. 



09 31 1129 



Tie Eurapam Pmnt OHtse is Irs ffO»)i^ iisbto lot these perikuias ^ft rn^^ Itor IIM IlitdMM Ot iRldrifi$32kin. 



I 

for mc» Oetaih abeut Ms OWIQK 9W Cfikial Jcun&l d Uw lfurd»»art PatO'M. pfiiSf. No. 1 2iB2 



8 



BEST AVAILABLE COPY 



